Recurrent events counted in evaluations of predictive accuracy

Objectives: To describe an alternate approach for the calculation of sensitivity and specificity when analyzing the accuracy of screening tools, which can be used when standard calculations may be inappropriate. Sensitivity^ER (ER denoting event rate) is the number of events correctly predicted, divided by the total number of events. Specificity^ER is the amount of time that study participants are predicted to be event negative, divided by the total amount of participant observed time. Variance estimates for these statistics are constructed by bootstrap resampling, taking into account event dependence.

Methods: Standard and alternate approaches for calculating sensitivity and specificity were applied to hospital falls risk screening tool data. In this application, the outcome of interest was a recurrent event, there were multiple applications of the screening tool, delays in screening tool  completion, and patients' follow-up durations were unequal.

Results: Application of sensitivity^ER and specificity^ER to this data not only provided a clearer description of the screening tool's overall accuracy, but also allowed examination of accuracy over time, accuracy in predicting specific event numbers, and evaluation of the added value that screening tool reapplications may have.

Conclusion: Sensitivity^ER and specificity^ER provide a valuable approach to screening tool evaluation in the clinical setting.

The impact of a line probe assay based diagnostic algorithm on time to treatment initiation and treatment outcomes for multidrug resistant TB patients in Arkhangelsk region, Russia

BACKGROUND: In the Arkhangelsk region of Northern Russia, multidrug-resistant (MDR) tuberculosis (TB) rates in new cases are amongst the highest in the world. In 2014, MDR-TB rates reached 31.7% among new cases and 56.9% among retreatment cases. The development of new diagnostic tools allows for faster detection of both TB and MDR-TB and should lead to reduced transmission by earlier initiation of anti-TB therapy. STUDY AIM: The PROVE-IT (Policy Relevant Outcomes from Validating Evidence on Impact) Russia study aimed to assess the impact of the implementation of line probe assay (LPA) as part of an LPA-based diagnostic algorithm for patients with presumptive MDR-TB focusing on time to treatment initiation with time from first-care seeking visit to the initiation of MDR-TB treatment rather than diagnostic accuracy as the primary outcome, and to assess treatment outcomes. We hypothesized that the implementation of LPA would result in faster time to treatment initiation and better treatment outcomes.

METHODS: A culture-based diagnostic algorithm used prior to LPA implementation was compared to an LPA-based algorithm that replaced BacTAlert and Löwenstein Jensen (LJ) for drug sensitivity testing. A total of 295 MDR-TB patients were included in the study, 163 diagnosed with the culture-based algorithm, 132 with the LPA-based algorithm.

RESULTS: Among smear positive patients, the implementation of the LPA-based algorithm was associated with a median decrease in time to MDR-TB treatment initiation of 50 and 66 days compared to the culture-based algorithm (BacTAlert and LJ respectively, p<0.001). In smear negative patients, the LPA-based algorithm was associated with a median decrease in time to MDR-TB treatment initiation of 78 days when compared to the culture-based algorithm (LJ, p<0.001). However, several weeks were still needed for treatment initiation in LPA-based algorithm, 24 days in smear positive, and 62 days in smear negative patients. Overall treatment outcomes were better in LPA-based algorithm compared to culture-based algorithm (p = 0.003). Treatment success rates at 20 months of treatment were higher in patients diagnosed with the LPA-based algorithm (65.2%) as compared to those diagnosed with the culture-based algorithm (44.8%). Mortality was also lower in the LPA-based algorithm group (7.6%) compared to the culture-based algorithm group (15.9%). There was no statistically significant difference in smear and culture conversion rates between the two algorithms.

CONCLUSION: The results of the study suggest that the introduction of LPA leads to faster time to MDR diagnosis and earlier treatment initiation as well as better treatment outcomes for patients with MDR-TB. These findings also highlight the need for further improvements within the health system to reduce both patient and diagnostic delays to truly optimize the impact of new, rapid diagnostics.

Role of knowledge and ability in student nurses` clinical decision-making

Little is known about the acquisition of decision-making skills in nursing students as a function of experience and academic ability. Knowing how experience and academic skills interact may help inform clinical education programs and formulate ways of assessing students' progress. The aims of the present study were to develop a problem-solving task capable of measuring clinical decision-making skills in novice nurses at different levels of domain-specific knowledge; and to establish the relative impact on decision-making of domain-specific knowledge and general ability as determinants of the acquisition of decision-making skills. Three types of clinical problems of increasing complexity were developed. Sixty second-year and third-year student nurses with high and low academic scores were studied in terms of their ability to generate hypotheses for a hypothetical case, recognize disconfirming information and the need to access additional information, and diagnostic accuracy. The results showed that general academic ability and knowledge function partly independently in the acquisition of expertise in nursing. Academic ability affects decision-making in low complexity tasks, but as case complexity increases, domain-specific knowledge and experience determines decision-making skills. There are important differences in the way novices with different levels of knowledge and ability make clinical decisions and these can be studied by systematically increasing the complexity of the decision task. These results have implications for the way in which clinical education is structured and evaluated.

Methodological considerations in the measurement of institutional and structural forms of HIV discrimination

The systematic measurement of HIV/AIDS-related discrimination is imperative within the current rhetoric that holds discrimination as one of the two ‘biggest' barriers to HIV/AIDS pandemic intervention. This paper provides a methodological critique of the UNAIDS (2000b) Protocol for the Identification of Discrimination against People Living with HIV (the Protocol) . Specifically, the paper focuses on the Protocol's capacity to accurately identify and measure institutional levels of HIV-related discrimination that allows data that are reliable and comparable across time and contexts. Conceptual issues including the Protocol's objective as an indicator versus a direct measure of discrimination and the role of the Protocol as a tool of research versus a tool of advocacy are explored. Design issues such as the operationalization of discrimination, appropriateness of indicator content, sampling and data collection strategies and issues of scoring are also evaluated. It is hoped that the matters outlined will provide readers with ways of critically reflecting and evaluating the findings of the research papers presented in this Special Issue, as well as pointing to ways of improving research design.

The performance of solarscan : an automated dermoscopy image analysis instrument for the diagnosis of primary melanoma

Objective To describe the diagnostic performance of SolarScan (Polartechnics Ltd, Sydney, Australia), an automated instrument for the diagnosis of primary melanoma.

Design Images from a data set of 2430 lesions (382 were melanomas; median Breslow thickness, 0.36 mm) were divided into a training set and an independent test set at a ratio of approximately 2:1. A diagnostic algorithm (absolute diagnosis of melanoma vs benign lesion and estimated probability of melanoma) was developed and its performance described on the test set. High-quality clinical and dermoscopy images with a detailed patient history for 78 lesions (13 of which were melanomas) from the test set were given to various clinicians to compare their diagnostic accuracy with that of SolarScan.

Setting Seven specialist referral centers and 2 general practice skin cancer clinics from 3 continents. Comparison between clinician diagnosis and SolarScan diagnosis was by 3 dermoscopy experts, 4 dermatologists, 3 trainee dermatologists, and 3 general practitioners.

Patients Images of the melanocytic lesions were obtained from patients who required either excision or digital monitoring to exclude malignancy.

Main Outcome Measures Sensitivity, specificity, the area under the receiver operator characteristic curve, median probability for the diagnosis of melanoma, a direct comparison of SolarScan with diagnoses performed by humans, and interinstrument and intrainstrument reproducibility.

Results The melanocytic-only diagnostic model was highly reproducible in the test set and gave a sensitivity of 91% (95% confidence interval [CI], 86%-96%) and specificity of 68% (95% CI, 64%-72%) for melanoma. SolarScan had comparable or superior sensitivity and specificity (85% vs 65%) compared with those of experts (90% vs 59%), dermatologists (81% vs 60%), trainees (85% vs 36%; P =.06), and general practitioners (62% vs 63%). The intraclass correlation coefficient of intrainstrument repeatability was 0.86 (95% CI, 0.83-0.88), indicating an excellent repeatability. There was no significant interinstrument variation (P = .80).

Conclusions SolarScan is a robust diagnostic instrument for pigmented or partially pigmented melanocytic lesions of the skin. Preliminary data suggest that its performance is comparable or superior to that of a range of clinician groups. However, these findings should be confirmed in a formal clinical trial.

A new instrument for targeting falls prevention interventions was accurate and clinically applicable in a hospital setting

Background and Objective: To describe the diagnostic accuracy and practical application of the Peter James Centre Falls Risk Assessment Tool (PJC-FRAT), a multidisciplinary falls risk screening and intervention deployment instrument.

Methods: In phase 1, the accuracy of the PJC-FRAT was prospectively compared to a gold standard (the STRATIFY) on a cohort of subacute hospital patients (n = 122). In phase 2, the PJC-FRAT was temporally reassessed using a subsequent cohort (n = 316), with results compared to those of phase 1. Primary outcomes were falls (events), fallers (patients who fell), and hospital completion rates of the PJC-FRAT.

Results: In phase 1, PJC-FRAT accuracy of identifying fallers showed  sensitivity of 73% (bootstrap 95% confidence interval CI = 55, 90) and specificity of 75% (95% CI = 66, 83), compared with the STRATIFY (cutoff ≥ 2/5) sensitivity of 77% (95% CI = 59, 92) and specificity of 51% (95% CI = 41, 61). This difference was not significant. In phase 2, accuracy of nursing staff using the PJC-FRAT was lower. PJC-FRAT completion rates varied among disciplines over both phases: nurses and physiotherapists, ≥90%; occupational therapists, ≥82%; and medical officers, ≥57%.

Conclusion: The PJC-FRAT was practical and relatively accurate as a predictor of falls and a deployment instrument for falls prevention interventions, although continued staff education may be necessary to maintain its accuracy.

Effect of bronchoalveolar lavage-directed therapy on pseudomonas aeruginosa infection and structural lung injury in children with cystic fibrosis

Context Early pulmonary infection in children with cystic fibrosis leads to increased morbidity and mortality. Despite wide use of oropharyngeal cultures to identify pulmonary infection, concerns remain over their diagnostic accuracy. While bronchoalveolar lavage (BAL) is an alternative diagnostic tool, evidence for its clinical benefit is lacking.

Objective To determine if BAL-directed therapy for pulmonary exacerbations during the first 5 years of life provides better outcomes than current standard practice relying on clinical features and oropharyngeal cultures.

Design, Setting, and Participants The Australasian Cystic Fibrosis Bronchoalveolar Lavage (ACFBAL) randomized controlled trial, recruiting infants diagnosed with cystic fibrosis through newborn screening programs in 8 Australasian cystic fibrosis centers. Recruitment occurred between June 1, 1999, and April 30, 2005, with the study ending on December 31, 2009.

Interventions BAL-directed (n=84) or standard (n=86) therapy until age 5 years. The BAL-directed therapy group underwent BAL before age 6 months when well, when hospitalized for pulmonary exacerbations, if Pseudomonas aeruginosa was detected in oropharyngeal specimens, and after P aeruginosa eradication therapy. Treatment was prescribed according to BAL or oropharyngeal culture results.

Main Outcome Measures Primary outcomes at age 5 years were prevalence of P aeruginosa on BAL cultures and total cystic fibrosis computed tomography (CF-CT) score (as a percentage of the maximum score) on high-resolution chest CT scan.

Results Of 267 infants diagnosed with cystic fibrosis following newborn screening, 170 were enrolled and randomized, and 157 completed the study. At age 5 years, 8 of 79 children (10%) in the BAL-directed therapy group and 9 of 76 (12%) in the standard therapy group had P aeruginosa in final BAL cultures (risk difference, −1.7% [95% confidence interval, −11.6% to 8.1%]; P=.73). Mean total CF-CT scores for the BAL-directed therapy and standard therapy groups were 3.0% and 2.8%, respectively (mean difference, 0.19% [95% confidence interval, −0.94% to 1.33%]; P=.74).

Conclusion Among infants diagnosed with cystic fibrosis, BAL-directed therapy did not result in a lower prevalence of P aeruginosa infection or lower total CF-CT score when compared with standard therapy at age 5 years.

Trial Registration anzctr.org.au Identifier: ACTRN12605000665639

Carpal tunnel syndrome: diagnostic usefulness of sonography

PURPOSE: To prospectively evaluate accuracy of sonography for diagnosis of carpal tunnel syndrome (CTS) in patients clinically suspected of having the disease in one or both hands.
MATERIALS AND METHODS: A prospective cohort of 133 patients suspected of having CTS were referred to a teaching hospital between October 2001 and June 2002 for electrodiagnostic study. One hundred twenty patients (98 women, 22 men; mean age, 49 years; range, 19–83 years) underwent sonography within 1 week after electrodiagnostic study. Radiologist was blinded to electrodiagnostic study results. Seventy-five patients had bilateral symptoms; 23 patients, right-hand symptoms; and 22 patients, left-hand symptoms (total, 195 symptomatic hands). Cross-sectional area of median nerve was measured at three levels: immediately proximal to carpal tunnel inlet, at carpal tunnel inlet, and at carpal tunnel outlet. Flexor retinaculum was used as a landmark to margins of carpal tunnel. Optimal threshold levels (determined with classification and regression tree analysis) for areas proximal to and at tunnel inlet and at tunnel outlet were used to discriminate between patients with and patients without disease. Sensitivity, specificity, and false-positive and false-negative rates were derived on the basis of final diagnosis, which was determined with clinical history and electrodiagnostic study results as reference standard.
RESULTS: For right hands, sonography had sensitivity of 94% (66 of 70); specificity, 65% (17 of 26); false-positive rate, 12% (nine of 75); and false-negative rate, 19% (four of 21) (cutoff, 0.09 cm2 proximal to tunnel inlet and 0.12 cm2 at tunnel outlet). For left hands, sensitivity was 83% (53 of 64); specificity, 73% (24 of 33); false-positive rate, 15% (nine of 62); and false-negative rate, 31% (11 of 35) (cutoff, 0.10 cm2 proximal to tunnel inlet).
CONCLUSION: Sonography is comparable to electrodiagnostic study in diagnosis of CTS and should be considered as initial test of choice for patients suspected of having CTS.

Sexing accuracy and indicators of maturation status in captive Murray cod Maccullochella peelii peelii using non-invasive ultrasonic imagery

Macroscopic- and histological-based assessments of gonad condition were compared with ultrasound images to determine the feasibility of this technology as a non-invasive diagnostic tool for identifying sex and assessing maturation status of Murray cod. Four age-classes (1⁺, 2⁺, 3⁺ and 6⁺ years), were sub-sampled at monthly intervals throughout their annual reproductive cycle and scanned with a 5 MHz linear transducer. An interpretation of sex was made from the resulting images and maximum cross-sectional gonad diameter and area were recorded. Fish were subsequently dissected to confirm gender, and the weights and maturation status of gonads determined and then compared with their respective image profile. Ovaries of females were usually a distinctive feature in ultrasound images, being particularly obvious in older and/or more developed fish. In contrast, the identification of male testis was more problematic. Nonetheless, identifying sex from ultrasound images was consistently achieved by recording the presence/absence of a female ovary (96% total sexing accuracy). Maximum cross-sectional ovary diameter and area were highly correlated with gonad weight (r² = 0.90 and 0.89, respectively) suggesting that indices of maturation status, comparable to the gonadosomatic index (GSI), can be obtained non-destructively from ultrasound scans of females. A less distinct relationship occurred between these dimensions and weight of testes (r² = 0.41). Significant increases (P < 0.05) in mean gonad index (GI, calculated from gonad diameter) occurred for most gonad development stages. However, differences in mean GI between maturation stages were confounded by phenotypic variability, indicating that GI may be limited to population level studies. Nevertheless, ultrasound images of ovaries at each development stage were visually distinctive and enabled qualitative evaluations of maturity, thereby complementing quantitative GI assessments. Repeated serial-monitoring of the same population using ultrasound appears to have great potential for tracking maturation-induced changes in broodfish.